Dataset is from Kaggle. link: https://www.kaggle.com/datasets/arjunbasandrai/25-indian-bird-species-with-226k-images
While researching for our dataset, we came across this really interesting dataset of different Indian Bird species which seemed exciting to work with, the reason for choosing this dataset. The goal of this ResNet50V2 bird classification project is to develop a highly accurate and effective model that can correctly identify various bird species from photos.
Due to the large number of bird species(we've taken 25 Indian species in our dataset), many of which have similar physical characteristics, classifying birds is a difficult undertaking. Yet, by learning from a sizable collection of bird photos and using deep learning models like ResNet50V2, it is feasible to create a model that can precisely identify between various bird species.
The model may use the pre-trained weights and architecture of the ResNet50V2 model and fine-tune them to perform well on the particular bird classification job by fine-tuning a pre-trained ResNet50V2 model on a dataset for bird classification. Instead of creating new CNN from start, we Will be using Microsoft's ResNet50V2 to fine-tune it according to our dataset.
Note: We have used Google for getting help with a few pieces of code as well as ChatGPT for debugging in certain places.
import tensorflow as tf
tf.__version__
'2.9.1'
from tensorflow.keras.layers import Input, Dense, Flatten
from tensorflow.keras.models import Model
from tensorflow.keras.preprocessing import image
from tensorflow.keras.preprocessing.image import ImageDataGenerator, load_img
from tensorflow.keras.models import Sequential
from tensorflow.keras.models import load_model
import numpy as np
from glob import glob
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import os
import random
from rl.agents.dqn import DQNAgent
from rl.policy import LinearAnnealedPolicy, BoltzmannQPolicy, EpsGreedyQPolicy
from rl.memory import SequentialMemory
from rl.core import Processor
import sklearn
# Import helper functions - https://gist.github.com/priteshkeleven/5756d60c56b3ac37c86cef348e76452f
from helper_functions import create_tensorboard_callback, plot_loss_curves, unzip_data, walk_through_dir, compare_historys
# Paths
train_dir = 'training_set'
test_dir = 'testing_set'
IMAGE_SIZE = [224, 224, 3]
# Setup data inputs
import tensorflow as tf
IMG_SIZE = (224, 224)
train_data = tf.keras.preprocessing.image_dataset_from_directory(train_dir,
image_size=IMG_SIZE,
label_mode="categorical",
batch_size=16)
test_data = tf.keras.preprocessing.image_dataset_from_directory(test_dir,
image_size=IMG_SIZE,
label_mode="categorical",
shuffle=False,
batch_size=16)
Found 16974 files belonging to 25 classes.
2023-03-28 16:23:34.936867: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node Your kernel may have been built without NUMA support. 2023-03-28 16:23:34.982309: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node Your kernel may have been built without NUMA support. 2023-03-28 16:23:34.982372: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node Your kernel may have been built without NUMA support. 2023-03-28 16:23:34.982973: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F AVX512_VNNI FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2023-03-28 16:23:34.985405: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node Your kernel may have been built without NUMA support. 2023-03-28 16:23:34.985441: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node Your kernel may have been built without NUMA support. 2023-03-28 16:23:34.985458: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node Your kernel may have been built without NUMA support. 2023-03-28 16:23:36.912465: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node Your kernel may have been built without NUMA support. 2023-03-28 16:23:36.912527: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node Your kernel may have been built without NUMA support. 2023-03-28 16:23:36.912533: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1616] Could not identify NUMA node of platform GPU id 0, defaulting to 0. Your kernel may not have been built with NUMA support. 2023-03-28 16:23:36.912557: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node Your kernel may have been built without NUMA support. 2023-03-28 16:23:36.912600: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3433 MB memory: -> device: 0, name: NVIDIA GeForce RTX 3060 Laptop GPU, pci bus id: 0000:01:00.0, compute capability: 8.6
Found 5647 files belonging to 25 classes.
# Creat checkpoint callback
checkpoint_path = "25_classes_birds_data_model_checkpoint/checkpoints.ckpt"
checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_path,
save_weights_only=True,
monitor="val_accuracy",
save_best_only=True)
# Create data augmentation layer
from tensorflow.keras import layers
from tensorflow.keras import Sequential
data_augmentation = Sequential([
layers.RandomFlip("horizontal"),
layers.RandomRotation(0.3),
layers.RandomHeight(0.3),
layers.RandomWidth(0.3),
layers.RandomZoom(0.3),
layers.Rescaling(1/255.)
], name="data_augmentation")
# View a random image and comapare it to augmentaed image
target_class = random.choice(train_data.class_names)
target_dir = "training_set/" + target_class
random_image = random.choice(os.listdir(target_dir))
random_image_path = target_dir + "/" + random_image
# Read in the random image
img = mpimg.imread(random_image_path)
plt.imshow(img)
plt.title(f"Original random image from class: {target_class}")
plt.axis(False);
# Plot augmented image
augmented_img = data_augmentation(img, training=True)
plt.figure()
plt.imshow(augmented_img)
plt.title(f"Augmented random image form class: {target_class}")
plt.axis(False);
base_model = tf.keras.applications.ResNet50V2(include_top=False)
base_model.trainable = False
base_model.summary()
Model: "resnet50v2"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, None, None, 0 []
3)]
conv1_pad (ZeroPadding2D) (None, None, None, 0 ['input_1[0][0]']
3)
conv1_conv (Conv2D) (None, None, None, 9472 ['conv1_pad[0][0]']
64)
pool1_pad (ZeroPadding2D) (None, None, None, 0 ['conv1_conv[0][0]']
64)
pool1_pool (MaxPooling2D) (None, None, None, 0 ['pool1_pad[0][0]']
64)
conv2_block1_preact_bn (BatchN (None, None, None, 256 ['pool1_pool[0][0]']
ormalization) 64)
conv2_block1_preact_relu (Acti (None, None, None, 0 ['conv2_block1_preact_bn[0][0]']
vation) 64)
conv2_block1_1_conv (Conv2D) (None, None, None, 4096 ['conv2_block1_preact_relu[0][0]'
64) ]
conv2_block1_1_bn (BatchNormal (None, None, None, 256 ['conv2_block1_1_conv[0][0]']
ization) 64)
conv2_block1_1_relu (Activatio (None, None, None, 0 ['conv2_block1_1_bn[0][0]']
n) 64)
conv2_block1_2_pad (ZeroPaddin (None, None, None, 0 ['conv2_block1_1_relu[0][0]']
g2D) 64)
conv2_block1_2_conv (Conv2D) (None, None, None, 36864 ['conv2_block1_2_pad[0][0]']
64)
conv2_block1_2_bn (BatchNormal (None, None, None, 256 ['conv2_block1_2_conv[0][0]']
ization) 64)
conv2_block1_2_relu (Activatio (None, None, None, 0 ['conv2_block1_2_bn[0][0]']
n) 64)
conv2_block1_0_conv (Conv2D) (None, None, None, 16640 ['conv2_block1_preact_relu[0][0]'
256) ]
conv2_block1_3_conv (Conv2D) (None, None, None, 16640 ['conv2_block1_2_relu[0][0]']
256)
conv2_block1_out (Add) (None, None, None, 0 ['conv2_block1_0_conv[0][0]',
256) 'conv2_block1_3_conv[0][0]']
conv2_block2_preact_bn (BatchN (None, None, None, 1024 ['conv2_block1_out[0][0]']
ormalization) 256)
conv2_block2_preact_relu (Acti (None, None, None, 0 ['conv2_block2_preact_bn[0][0]']
vation) 256)
conv2_block2_1_conv (Conv2D) (None, None, None, 16384 ['conv2_block2_preact_relu[0][0]'
64) ]
conv2_block2_1_bn (BatchNormal (None, None, None, 256 ['conv2_block2_1_conv[0][0]']
ization) 64)
conv2_block2_1_relu (Activatio (None, None, None, 0 ['conv2_block2_1_bn[0][0]']
n) 64)
conv2_block2_2_pad (ZeroPaddin (None, None, None, 0 ['conv2_block2_1_relu[0][0]']
g2D) 64)
conv2_block2_2_conv (Conv2D) (None, None, None, 36864 ['conv2_block2_2_pad[0][0]']
64)
conv2_block2_2_bn (BatchNormal (None, None, None, 256 ['conv2_block2_2_conv[0][0]']
ization) 64)
conv2_block2_2_relu (Activatio (None, None, None, 0 ['conv2_block2_2_bn[0][0]']
n) 64)
conv2_block2_3_conv (Conv2D) (None, None, None, 16640 ['conv2_block2_2_relu[0][0]']
256)
conv2_block2_out (Add) (None, None, None, 0 ['conv2_block1_out[0][0]',
256) 'conv2_block2_3_conv[0][0]']
conv2_block3_preact_bn (BatchN (None, None, None, 1024 ['conv2_block2_out[0][0]']
ormalization) 256)
conv2_block3_preact_relu (Acti (None, None, None, 0 ['conv2_block3_preact_bn[0][0]']
vation) 256)
conv2_block3_1_conv (Conv2D) (None, None, None, 16384 ['conv2_block3_preact_relu[0][0]'
64) ]
conv2_block3_1_bn (BatchNormal (None, None, None, 256 ['conv2_block3_1_conv[0][0]']
ization) 64)
conv2_block3_1_relu (Activatio (None, None, None, 0 ['conv2_block3_1_bn[0][0]']
n) 64)
conv2_block3_2_pad (ZeroPaddin (None, None, None, 0 ['conv2_block3_1_relu[0][0]']
g2D) 64)
conv2_block3_2_conv (Conv2D) (None, None, None, 36864 ['conv2_block3_2_pad[0][0]']
64)
conv2_block3_2_bn (BatchNormal (None, None, None, 256 ['conv2_block3_2_conv[0][0]']
ization) 64)
conv2_block3_2_relu (Activatio (None, None, None, 0 ['conv2_block3_2_bn[0][0]']
n) 64)
max_pooling2d (MaxPooling2D) (None, None, None, 0 ['conv2_block2_out[0][0]']
256)
conv2_block3_3_conv (Conv2D) (None, None, None, 16640 ['conv2_block3_2_relu[0][0]']
256)
conv2_block3_out (Add) (None, None, None, 0 ['max_pooling2d[0][0]',
256) 'conv2_block3_3_conv[0][0]']
conv3_block1_preact_bn (BatchN (None, None, None, 1024 ['conv2_block3_out[0][0]']
ormalization) 256)
conv3_block1_preact_relu (Acti (None, None, None, 0 ['conv3_block1_preact_bn[0][0]']
vation) 256)
conv3_block1_1_conv (Conv2D) (None, None, None, 32768 ['conv3_block1_preact_relu[0][0]'
128) ]
conv3_block1_1_bn (BatchNormal (None, None, None, 512 ['conv3_block1_1_conv[0][0]']
ization) 128)
conv3_block1_1_relu (Activatio (None, None, None, 0 ['conv3_block1_1_bn[0][0]']
n) 128)
conv3_block1_2_pad (ZeroPaddin (None, None, None, 0 ['conv3_block1_1_relu[0][0]']
g2D) 128)
conv3_block1_2_conv (Conv2D) (None, None, None, 147456 ['conv3_block1_2_pad[0][0]']
128)
conv3_block1_2_bn (BatchNormal (None, None, None, 512 ['conv3_block1_2_conv[0][0]']
ization) 128)
conv3_block1_2_relu (Activatio (None, None, None, 0 ['conv3_block1_2_bn[0][0]']
n) 128)
conv3_block1_0_conv (Conv2D) (None, None, None, 131584 ['conv3_block1_preact_relu[0][0]'
512) ]
conv3_block1_3_conv (Conv2D) (None, None, None, 66048 ['conv3_block1_2_relu[0][0]']
512)
conv3_block1_out (Add) (None, None, None, 0 ['conv3_block1_0_conv[0][0]',
512) 'conv3_block1_3_conv[0][0]']
conv3_block2_preact_bn (BatchN (None, None, None, 2048 ['conv3_block1_out[0][0]']
ormalization) 512)
conv3_block2_preact_relu (Acti (None, None, None, 0 ['conv3_block2_preact_bn[0][0]']
vation) 512)
conv3_block2_1_conv (Conv2D) (None, None, None, 65536 ['conv3_block2_preact_relu[0][0]'
128) ]
conv3_block2_1_bn (BatchNormal (None, None, None, 512 ['conv3_block2_1_conv[0][0]']
ization) 128)
conv3_block2_1_relu (Activatio (None, None, None, 0 ['conv3_block2_1_bn[0][0]']
n) 128)
conv3_block2_2_pad (ZeroPaddin (None, None, None, 0 ['conv3_block2_1_relu[0][0]']
g2D) 128)
conv3_block2_2_conv (Conv2D) (None, None, None, 147456 ['conv3_block2_2_pad[0][0]']
128)
conv3_block2_2_bn (BatchNormal (None, None, None, 512 ['conv3_block2_2_conv[0][0]']
ization) 128)
conv3_block2_2_relu (Activatio (None, None, None, 0 ['conv3_block2_2_bn[0][0]']
n) 128)
conv3_block2_3_conv (Conv2D) (None, None, None, 66048 ['conv3_block2_2_relu[0][0]']
512)
conv3_block2_out (Add) (None, None, None, 0 ['conv3_block1_out[0][0]',
512) 'conv3_block2_3_conv[0][0]']
conv3_block3_preact_bn (BatchN (None, None, None, 2048 ['conv3_block2_out[0][0]']
ormalization) 512)
conv3_block3_preact_relu (Acti (None, None, None, 0 ['conv3_block3_preact_bn[0][0]']
vation) 512)
conv3_block3_1_conv (Conv2D) (None, None, None, 65536 ['conv3_block3_preact_relu[0][0]'
128) ]
conv3_block3_1_bn (BatchNormal (None, None, None, 512 ['conv3_block3_1_conv[0][0]']
ization) 128)
conv3_block3_1_relu (Activatio (None, None, None, 0 ['conv3_block3_1_bn[0][0]']
n) 128)
conv3_block3_2_pad (ZeroPaddin (None, None, None, 0 ['conv3_block3_1_relu[0][0]']
g2D) 128)
conv3_block3_2_conv (Conv2D) (None, None, None, 147456 ['conv3_block3_2_pad[0][0]']
128)
conv3_block3_2_bn (BatchNormal (None, None, None, 512 ['conv3_block3_2_conv[0][0]']
ization) 128)
conv3_block3_2_relu (Activatio (None, None, None, 0 ['conv3_block3_2_bn[0][0]']
n) 128)
conv3_block3_3_conv (Conv2D) (None, None, None, 66048 ['conv3_block3_2_relu[0][0]']
512)
conv3_block3_out (Add) (None, None, None, 0 ['conv3_block2_out[0][0]',
512) 'conv3_block3_3_conv[0][0]']
conv3_block4_preact_bn (BatchN (None, None, None, 2048 ['conv3_block3_out[0][0]']
ormalization) 512)
conv3_block4_preact_relu (Acti (None, None, None, 0 ['conv3_block4_preact_bn[0][0]']
vation) 512)
conv3_block4_1_conv (Conv2D) (None, None, None, 65536 ['conv3_block4_preact_relu[0][0]'
128) ]
conv3_block4_1_bn (BatchNormal (None, None, None, 512 ['conv3_block4_1_conv[0][0]']
ization) 128)
conv3_block4_1_relu (Activatio (None, None, None, 0 ['conv3_block4_1_bn[0][0]']
n) 128)
conv3_block4_2_pad (ZeroPaddin (None, None, None, 0 ['conv3_block4_1_relu[0][0]']
g2D) 128)
conv3_block4_2_conv (Conv2D) (None, None, None, 147456 ['conv3_block4_2_pad[0][0]']
128)
conv3_block4_2_bn (BatchNormal (None, None, None, 512 ['conv3_block4_2_conv[0][0]']
ization) 128)
conv3_block4_2_relu (Activatio (None, None, None, 0 ['conv3_block4_2_bn[0][0]']
n) 128)
max_pooling2d_1 (MaxPooling2D) (None, None, None, 0 ['conv3_block3_out[0][0]']
512)
conv3_block4_3_conv (Conv2D) (None, None, None, 66048 ['conv3_block4_2_relu[0][0]']
512)
conv3_block4_out (Add) (None, None, None, 0 ['max_pooling2d_1[0][0]',
512) 'conv3_block4_3_conv[0][0]']
conv4_block1_preact_bn (BatchN (None, None, None, 2048 ['conv3_block4_out[0][0]']
ormalization) 512)
conv4_block1_preact_relu (Acti (None, None, None, 0 ['conv4_block1_preact_bn[0][0]']
vation) 512)
conv4_block1_1_conv (Conv2D) (None, None, None, 131072 ['conv4_block1_preact_relu[0][0]'
256) ]
conv4_block1_1_bn (BatchNormal (None, None, None, 1024 ['conv4_block1_1_conv[0][0]']
ization) 256)
conv4_block1_1_relu (Activatio (None, None, None, 0 ['conv4_block1_1_bn[0][0]']
n) 256)
conv4_block1_2_pad (ZeroPaddin (None, None, None, 0 ['conv4_block1_1_relu[0][0]']
g2D) 256)
conv4_block1_2_conv (Conv2D) (None, None, None, 589824 ['conv4_block1_2_pad[0][0]']
256)
conv4_block1_2_bn (BatchNormal (None, None, None, 1024 ['conv4_block1_2_conv[0][0]']
ization) 256)
conv4_block1_2_relu (Activatio (None, None, None, 0 ['conv4_block1_2_bn[0][0]']
n) 256)
conv4_block1_0_conv (Conv2D) (None, None, None, 525312 ['conv4_block1_preact_relu[0][0]'
1024) ]
conv4_block1_3_conv (Conv2D) (None, None, None, 263168 ['conv4_block1_2_relu[0][0]']
1024)
conv4_block1_out (Add) (None, None, None, 0 ['conv4_block1_0_conv[0][0]',
1024) 'conv4_block1_3_conv[0][0]']
conv4_block2_preact_bn (BatchN (None, None, None, 4096 ['conv4_block1_out[0][0]']
ormalization) 1024)
conv4_block2_preact_relu (Acti (None, None, None, 0 ['conv4_block2_preact_bn[0][0]']
vation) 1024)
conv4_block2_1_conv (Conv2D) (None, None, None, 262144 ['conv4_block2_preact_relu[0][0]'
256) ]
conv4_block2_1_bn (BatchNormal (None, None, None, 1024 ['conv4_block2_1_conv[0][0]']
ization) 256)
conv4_block2_1_relu (Activatio (None, None, None, 0 ['conv4_block2_1_bn[0][0]']
n) 256)
conv4_block2_2_pad (ZeroPaddin (None, None, None, 0 ['conv4_block2_1_relu[0][0]']
g2D) 256)
conv4_block2_2_conv (Conv2D) (None, None, None, 589824 ['conv4_block2_2_pad[0][0]']
256)
conv4_block2_2_bn (BatchNormal (None, None, None, 1024 ['conv4_block2_2_conv[0][0]']
ization) 256)
conv4_block2_2_relu (Activatio (None, None, None, 0 ['conv4_block2_2_bn[0][0]']
n) 256)
conv4_block2_3_conv (Conv2D) (None, None, None, 263168 ['conv4_block2_2_relu[0][0]']
1024)
conv4_block2_out (Add) (None, None, None, 0 ['conv4_block1_out[0][0]',
1024) 'conv4_block2_3_conv[0][0]']
conv4_block3_preact_bn (BatchN (None, None, None, 4096 ['conv4_block2_out[0][0]']
ormalization) 1024)
conv4_block3_preact_relu (Acti (None, None, None, 0 ['conv4_block3_preact_bn[0][0]']
vation) 1024)
conv4_block3_1_conv (Conv2D) (None, None, None, 262144 ['conv4_block3_preact_relu[0][0]'
256) ]
conv4_block3_1_bn (BatchNormal (None, None, None, 1024 ['conv4_block3_1_conv[0][0]']
ization) 256)
conv4_block3_1_relu (Activatio (None, None, None, 0 ['conv4_block3_1_bn[0][0]']
n) 256)
conv4_block3_2_pad (ZeroPaddin (None, None, None, 0 ['conv4_block3_1_relu[0][0]']
g2D) 256)
conv4_block3_2_conv (Conv2D) (None, None, None, 589824 ['conv4_block3_2_pad[0][0]']
256)
conv4_block3_2_bn (BatchNormal (None, None, None, 1024 ['conv4_block3_2_conv[0][0]']
ization) 256)
conv4_block3_2_relu (Activatio (None, None, None, 0 ['conv4_block3_2_bn[0][0]']
n) 256)
conv4_block3_3_conv (Conv2D) (None, None, None, 263168 ['conv4_block3_2_relu[0][0]']
1024)
conv4_block3_out (Add) (None, None, None, 0 ['conv4_block2_out[0][0]',
1024) 'conv4_block3_3_conv[0][0]']
conv4_block4_preact_bn (BatchN (None, None, None, 4096 ['conv4_block3_out[0][0]']
ormalization) 1024)
conv4_block4_preact_relu (Acti (None, None, None, 0 ['conv4_block4_preact_bn[0][0]']
vation) 1024)
conv4_block4_1_conv (Conv2D) (None, None, None, 262144 ['conv4_block4_preact_relu[0][0]'
256) ]
conv4_block4_1_bn (BatchNormal (None, None, None, 1024 ['conv4_block4_1_conv[0][0]']
ization) 256)
conv4_block4_1_relu (Activatio (None, None, None, 0 ['conv4_block4_1_bn[0][0]']
n) 256)
conv4_block4_2_pad (ZeroPaddin (None, None, None, 0 ['conv4_block4_1_relu[0][0]']
g2D) 256)
conv4_block4_2_conv (Conv2D) (None, None, None, 589824 ['conv4_block4_2_pad[0][0]']
256)
conv4_block4_2_bn (BatchNormal (None, None, None, 1024 ['conv4_block4_2_conv[0][0]']
ization) 256)
conv4_block4_2_relu (Activatio (None, None, None, 0 ['conv4_block4_2_bn[0][0]']
n) 256)
conv4_block4_3_conv (Conv2D) (None, None, None, 263168 ['conv4_block4_2_relu[0][0]']
1024)
conv4_block4_out (Add) (None, None, None, 0 ['conv4_block3_out[0][0]',
1024) 'conv4_block4_3_conv[0][0]']
conv4_block5_preact_bn (BatchN (None, None, None, 4096 ['conv4_block4_out[0][0]']
ormalization) 1024)
conv4_block5_preact_relu (Acti (None, None, None, 0 ['conv4_block5_preact_bn[0][0]']
vation) 1024)
conv4_block5_1_conv (Conv2D) (None, None, None, 262144 ['conv4_block5_preact_relu[0][0]'
256) ]
conv4_block5_1_bn (BatchNormal (None, None, None, 1024 ['conv4_block5_1_conv[0][0]']
ization) 256)
conv4_block5_1_relu (Activatio (None, None, None, 0 ['conv4_block5_1_bn[0][0]']
n) 256)
conv4_block5_2_pad (ZeroPaddin (None, None, None, 0 ['conv4_block5_1_relu[0][0]']
g2D) 256)
conv4_block5_2_conv (Conv2D) (None, None, None, 589824 ['conv4_block5_2_pad[0][0]']
256)
conv4_block5_2_bn (BatchNormal (None, None, None, 1024 ['conv4_block5_2_conv[0][0]']
ization) 256)
conv4_block5_2_relu (Activatio (None, None, None, 0 ['conv4_block5_2_bn[0][0]']
n) 256)
conv4_block5_3_conv (Conv2D) (None, None, None, 263168 ['conv4_block5_2_relu[0][0]']
1024)
conv4_block5_out (Add) (None, None, None, 0 ['conv4_block4_out[0][0]',
1024) 'conv4_block5_3_conv[0][0]']
conv4_block6_preact_bn (BatchN (None, None, None, 4096 ['conv4_block5_out[0][0]']
ormalization) 1024)
conv4_block6_preact_relu (Acti (None, None, None, 0 ['conv4_block6_preact_bn[0][0]']
vation) 1024)
conv4_block6_1_conv (Conv2D) (None, None, None, 262144 ['conv4_block6_preact_relu[0][0]'
256) ]
conv4_block6_1_bn (BatchNormal (None, None, None, 1024 ['conv4_block6_1_conv[0][0]']
ization) 256)
conv4_block6_1_relu (Activatio (None, None, None, 0 ['conv4_block6_1_bn[0][0]']
n) 256)
conv4_block6_2_pad (ZeroPaddin (None, None, None, 0 ['conv4_block6_1_relu[0][0]']
g2D) 256)
conv4_block6_2_conv (Conv2D) (None, None, None, 589824 ['conv4_block6_2_pad[0][0]']
256)
conv4_block6_2_bn (BatchNormal (None, None, None, 1024 ['conv4_block6_2_conv[0][0]']
ization) 256)
conv4_block6_2_relu (Activatio (None, None, None, 0 ['conv4_block6_2_bn[0][0]']
n) 256)
max_pooling2d_2 (MaxPooling2D) (None, None, None, 0 ['conv4_block5_out[0][0]']
1024)
conv4_block6_3_conv (Conv2D) (None, None, None, 263168 ['conv4_block6_2_relu[0][0]']
1024)
conv4_block6_out (Add) (None, None, None, 0 ['max_pooling2d_2[0][0]',
1024) 'conv4_block6_3_conv[0][0]']
conv5_block1_preact_bn (BatchN (None, None, None, 4096 ['conv4_block6_out[0][0]']
ormalization) 1024)
conv5_block1_preact_relu (Acti (None, None, None, 0 ['conv5_block1_preact_bn[0][0]']
vation) 1024)
conv5_block1_1_conv (Conv2D) (None, None, None, 524288 ['conv5_block1_preact_relu[0][0]'
512) ]
conv5_block1_1_bn (BatchNormal (None, None, None, 2048 ['conv5_block1_1_conv[0][0]']
ization) 512)
conv5_block1_1_relu (Activatio (None, None, None, 0 ['conv5_block1_1_bn[0][0]']
n) 512)
conv5_block1_2_pad (ZeroPaddin (None, None, None, 0 ['conv5_block1_1_relu[0][0]']
g2D) 512)
conv5_block1_2_conv (Conv2D) (None, None, None, 2359296 ['conv5_block1_2_pad[0][0]']
512)
conv5_block1_2_bn (BatchNormal (None, None, None, 2048 ['conv5_block1_2_conv[0][0]']
ization) 512)
conv5_block1_2_relu (Activatio (None, None, None, 0 ['conv5_block1_2_bn[0][0]']
n) 512)
conv5_block1_0_conv (Conv2D) (None, None, None, 2099200 ['conv5_block1_preact_relu[0][0]'
2048) ]
conv5_block1_3_conv (Conv2D) (None, None, None, 1050624 ['conv5_block1_2_relu[0][0]']
2048)
conv5_block1_out (Add) (None, None, None, 0 ['conv5_block1_0_conv[0][0]',
2048) 'conv5_block1_3_conv[0][0]']
conv5_block2_preact_bn (BatchN (None, None, None, 8192 ['conv5_block1_out[0][0]']
ormalization) 2048)
conv5_block2_preact_relu (Acti (None, None, None, 0 ['conv5_block2_preact_bn[0][0]']
vation) 2048)
conv5_block2_1_conv (Conv2D) (None, None, None, 1048576 ['conv5_block2_preact_relu[0][0]'
512) ]
conv5_block2_1_bn (BatchNormal (None, None, None, 2048 ['conv5_block2_1_conv[0][0]']
ization) 512)
conv5_block2_1_relu (Activatio (None, None, None, 0 ['conv5_block2_1_bn[0][0]']
n) 512)
conv5_block2_2_pad (ZeroPaddin (None, None, None, 0 ['conv5_block2_1_relu[0][0]']
g2D) 512)
conv5_block2_2_conv (Conv2D) (None, None, None, 2359296 ['conv5_block2_2_pad[0][0]']
512)
conv5_block2_2_bn (BatchNormal (None, None, None, 2048 ['conv5_block2_2_conv[0][0]']
ization) 512)
conv5_block2_2_relu (Activatio (None, None, None, 0 ['conv5_block2_2_bn[0][0]']
n) 512)
conv5_block2_3_conv (Conv2D) (None, None, None, 1050624 ['conv5_block2_2_relu[0][0]']
2048)
conv5_block2_out (Add) (None, None, None, 0 ['conv5_block1_out[0][0]',
2048) 'conv5_block2_3_conv[0][0]']
conv5_block3_preact_bn (BatchN (None, None, None, 8192 ['conv5_block2_out[0][0]']
ormalization) 2048)
conv5_block3_preact_relu (Acti (None, None, None, 0 ['conv5_block3_preact_bn[0][0]']
vation) 2048)
conv5_block3_1_conv (Conv2D) (None, None, None, 1048576 ['conv5_block3_preact_relu[0][0]'
512) ]
conv5_block3_1_bn (BatchNormal (None, None, None, 2048 ['conv5_block3_1_conv[0][0]']
ization) 512)
conv5_block3_1_relu (Activatio (None, None, None, 0 ['conv5_block3_1_bn[0][0]']
n) 512)
conv5_block3_2_pad (ZeroPaddin (None, None, None, 0 ['conv5_block3_1_relu[0][0]']
g2D) 512)
conv5_block3_2_conv (Conv2D) (None, None, None, 2359296 ['conv5_block3_2_pad[0][0]']
512)
conv5_block3_2_bn (BatchNormal (None, None, None, 2048 ['conv5_block3_2_conv[0][0]']
ization) 512)
conv5_block3_2_relu (Activatio (None, None, None, 0 ['conv5_block3_2_bn[0][0]']
n) 512)
conv5_block3_3_conv (Conv2D) (None, None, None, 1050624 ['conv5_block3_2_relu[0][0]']
2048)
conv5_block3_out (Add) (None, None, None, 0 ['conv5_block2_out[0][0]',
2048) 'conv5_block3_3_conv[0][0]']
post_bn (BatchNormalization) (None, None, None, 8192 ['conv5_block3_out[0][0]']
2048)
post_relu (Activation) (None, None, None, 0 ['post_bn[0][0]']
2048)
==================================================================================================
Total params: 23,564,800
Trainable params: 0
Non-trainable params: 23,564,800
__________________________________________________________________________________________________
# Setup model architecture with trainable top layres
inputs = layers.Input(shape=(224, 224, 3), name="input_layer")
x = data_augmentation(inputs) # Augment images during training
x = base_model(x, training=False) # put base model in inference mode to set weights frozen
x = layers.GlobalAveragePooling2D(name="global_avg_pool_layer")(x)
outputs = layers.Dense(len(train_data.class_names), activation="softmax", name="output_layer")(x)
model = tf.keras.Model(inputs, outputs)
# Compile the model
model.compile(loss="categorical_crossentropy",
optimizer=tf.keras.optimizers.Adam(),
metrics=["accuracy"])
model.summary()
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_layer (InputLayer) [(None, 224, 224, 3)] 0
data_augmentation (Sequenti (None, 224, 224, 3) 0
al)
resnet50v2 (Functional) (None, None, None, 2048) 23564800
global_avg_pool_layer (Glob (None, 2048) 0
alAveragePooling2D)
output_layer (Dense) (None, 25) 51225
=================================================================
Total params: 23,616,025
Trainable params: 51,225
Non-trainable params: 23,564,800
_________________________________________________________________
# Fit the model
base_model_history = model.fit(train_data,
epochs=5,
validation_data=test_data,
validation_steps=int(0.15*len(test_data)),
callbacks=[checkpoint_callback, create_tensorboard_callback(dir_name="transfer_learning", experiment_name="base_model")])
model = tf.keras.models.load_model('bird_clf_resnet50v2_base_model.h5')
# Evaluate on whole test dataset
base_model_results = model.evaluate(test_data)
base_model_results
239/353 [===================>..........] - ETA: 4s - loss: 0.6411 - accuracy: 0.8324
2023-03-25 22:07:56.981868: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
250/353 [====================>.........] - ETA: 4s - loss: 0.6377 - accuracy: 0.8315
2023-03-25 22:07:57.382174: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile 2023-03-25 22:07:57.382757: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
317/353 [=========================>....] - ETA: 1s - loss: 0.5826 - accuracy: 0.8452
2023-03-25 22:07:59.938045: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
353/353 [==============================] - 15s 41ms/step - loss: 0.5703 - accuracy: 0.8505
[0.5703216195106506, 0.8505401015281677]
We plot loss and accuracy curves, comparing trainin and validation data. We observe for the first epoch, trend was opposite of expectation but then starts to improve gradually.
plot_loss_curves(base_model_history)
model.save('bird_clf_resnet50v2_base_model.h5')
# unfreeze all of the layers in the base model
base_model.trainable = True
# Refreeze every layer except last 5
for layer in base_model.layers[:-5]:
layer.trainable = False
for i, layer in enumerate(base_model.layers[-10:]):
print(i, layer.name, layer.trainable)
0 conv5_block3_1_bn False 1 conv5_block3_1_relu False 2 conv5_block3_2_pad False 3 conv5_block3_2_conv False 4 conv5_block3_2_bn False 5 conv5_block3_2_relu True 6 conv5_block3_3_conv True 7 conv5_block3_out True 8 post_bn True 9 post_relu True
# Recompiling model with lower learning_rate
model.compile(loss="categorical_crossentropy",
optimizer=tf.keras.optimizers.Adam(learning_rate=0.0001),
metrics=["accuracy"])
model.summary()
Model: "model_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_layer (InputLayer) [(None, 224, 224, 3)] 0
data_augmentation (Sequenti (None, 224, 224, 3) 0
al)
resnet50v2 (Functional) (None, None, None, 2048) 23564800
global_avg_pool_layer (Glob (None, 2048) 0
alAveragePooling2D)
output_layer (Dense) (None, 25) 51225
=================================================================
Total params: 23,616,025
Trainable params: 1,105,945
Non-trainable params: 22,510,080
_________________________________________________________________
# Fit the model again
initial_epochs = 5
history_fine_tuned_5_layers_base_model = model.fit(train_data,
epochs=initial_epochs + 5,
validation_data=test_data,
validation_steps=int(0.15 * len(test_data)),
initial_epoch=base_model_history.epoch[-1],
callbacks=[create_tensorboard_callback(dir_name="transfer_learning",
experiment_name="history_fine_tuned_5_layers_base_model")])
# Evaluate on whole test dataset
fine_tune_5_layers_results = model.evaluate(test_data)
fine_tune_5_layers_results
238/353 [===================>..........] - ETA: 4s - loss: 0.5325 - accuracy: 0.8564
2023-03-25 22:42:12.235521: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
250/353 [====================>.........] - ETA: 4s - loss: 0.5237 - accuracy: 0.8580
2023-03-25 22:42:12.702094: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile 2023-03-25 22:42:12.703044: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
318/353 [==========================>...] - ETA: 1s - loss: 0.4868 - accuracy: 0.8667
2023-03-25 22:42:15.542373: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
353/353 [==============================] - 15s 42ms/step - loss: 0.4837 - accuracy: 0.8697
[0.4836820960044861, 0.8696653246879578]
model = tf.keras.models.load_model('bird_clf_resnet50v2_fine_tuned_5_layers.h5')
compare_historys(original_history=base_model_history,
new_history=history_fine_tuned_5_layers_base_model,
initial_epochs=5)
model = tf.keras.models.load_model("bird_clf_resnet50v2_fine_tuned_5_layers.h5")
# Save it as a h5 file
model.save('bird_clf_resnet50v2_fine_tuned_5_layers.h5')
We are not able to run reinforcement learning since DQN agent requires only 1 output from our fine-tuned model while it gives list of 25 which is not supported by DQN agent. (We tried solving this but because of time constraints, we were unable to fix it. We would try to fix it in our project-2 which will be an extension of this one.)
INPUT_SHAPE = (224, 224, 3)
class ClassifyProcessor(Processor):
def process_observation(self, observation):
img = observation.reshape(INPUT_SHAPE)
processed_observation = np.array(img)
return processed_observation
def process_state_batch(self, batch):
batch = batch.reshape((-1,) + INPUT_SHAPE)
processed_batch = batch.astype('float32') / 1.
return processed_batch
def process_reward(self, reward):
return np.clip(reward, -1., 1.)
memory = SequentialMemory(limit=100000, window_length=1)
processor = ClassifyProcessor()
policy = LinearAnnealedPolicy(EpsGreedyQPolicy(), attr='eps', value_max=1., value_min=.1, value_test=.05,
nb_steps=100000)
model.summary()
Model: "model_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_layer (InputLayer) [(None, 224, 224, 3)] 0
data_augmentation (Sequenti (None, 224, 224, 3) 0
al)
resnet50v2 (Functional) (None, None, None, 2048) 23564800
global_avg_pool_layer (Glob (None, 2048) 0
alAveragePooling2D)
output_layer (Dense) (None, 25) 51225
=================================================================
Total params: 23,616,025
Trainable params: 1,105,945
Non-trainable params: 22,510,080
_________________________________________________________________
dqn = DQNAgent(model=model, nb_actions=len(test_data.class_names), policy=policy, memory=memory,
processor=processor, nb_steps_warmup=50000, gamma=0.5, target_model_update=10000,
train_interval=4, delta_clip=1.)
dqn.compile(Adam(learning_rate=.00025), metrics=['mae'])
dqn.fit(env, nb_steps=training_steps, log_interval=60000)
--------------------------------------------------------------------------- TypeError Traceback (most recent call last) Cell In[127], line 1 ----> 1 dqn = DQNAgent(model=model, nb_actions=len(test_data.class_names), policy=policy, memory=memory, 2 processor=processor, nb_steps_warmup=50000, gamma=0.5, target_model_update=10000, 3 train_interval=4, delta_clip=1.) 4 dqn.compile(Adam(learning_rate=.00025), metrics=['mae']) 6 dqn.fit(env, nb_steps=training_steps, log_interval=60000) File ~/miniconda3/envs/hf/lib/python3.10/site-packages/rl/agents/dqn.py:108, in DQNAgent.__init__(self, model, policy, test_policy, enable_double_dqn, enable_dueling_network, dueling_type, *args, **kwargs) 105 super(DQNAgent, self).__init__(*args, **kwargs) 107 # Validate (important) input. --> 108 if hasattr(model.output, '__len__') and len(model.output) > 1: 109 raise ValueError('Model "{}" has more than one output. DQN expects a model that has a single output.'.format(model)) 110 if model.output._keras_shape != (None, self.nb_actions): File ~/miniconda3/envs/hf/lib/python3.10/site-packages/keras/engine/keras_tensor.py:221, in KerasTensor.__len__(self) 220 def __len__(self): --> 221 raise TypeError('Keras symbolic inputs/outputs do not ' 222 'implement `__len__`. You may be ' 223 'trying to pass Keras symbolic inputs/outputs ' 224 'to a TF API that does not register dispatching, ' 225 'preventing Keras from automatically ' 226 'converting the API call to a lambda layer ' 227 'in the Functional Model. This error will also get raised ' 228 'if you try asserting a symbolic input/output directly.') TypeError: Keras symbolic inputs/outputs do not implement `__len__`. You may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model. This error will also get raised if you try asserting a symbolic input/output directly.
Error - Model is not accepting input shape of image. It's getting converted in between layers.
import cv2
def preprocess_image(image_path):
try:
# Load the image from the specified path
image = cv2.imread(image_path)
# Check if the image is None or empty
if image is None or image.size == 0:
raise ValueError(f"Failed to read image from {image_path}")
# Resize the image to a fixed size
image = cv2.resize(image, (224, 224))
# Convert the image to floating point format
image = image.astype('float32')
# Normalize the image by dividing by 255
image /= 255
return image
except Exception as e:
print(image_path)
print(f"Error processing image {image_path}: {str(e)}")
return None
# Define a custom gym environment
class CustomEnv(gym.Env):
def __init__(self):
self.train_dir = "training_set"
self.test_dir = "testing_set"
self.training_mode = True
self.observation_space = gym.spaces.Box(
low=0,
high=255,
shape=(224, 224, 3),
dtype='uint8'
)
self.action_space = gym.spaces.Box(
low=-1,
high=1,
shape=(1,),
dtype='float32'
)
# Initialize ResNet50V2 model
self.model = model
# Unfreeze last 5 layers for fine-tuning
for layer in self.model.layers[-5:]:
layer.trainable = True
# Add top layers for action prediction
x = Flatten()(self.model.output)
x = Dense(512, activation='relu')(x)
self.output_layer = Dense(self.action_space.shape[0])(x)
self.model = Model(inputs=self.model.input, outputs=self.output_layer)
# Define optimizer
self.optimizer = Adam(learning_rate=0.001)
# Define loss function
self.loss_fn = tf.keras.losses.MeanSquaredError()
self.current_observation = None
self.current_step = 0
self.max_steps = 1000
self.reward_range = (-float('inf'), float('inf'))
def _get_observation(self):
if self.training_mode:
# Get the path to a random image in the train directory
class_dir = np.random.choice(os.listdir(self.train_dir))
image_file = os.path.join(self.train_dir, class_dir, np.random.choice(os.listdir(os.path.join(train_dir, class_dir))))
else:
# Get the path to the next image in the test directory
class_dir = np.random.choice(os.listdir(self.test_dir))
image_file = os.path.join(self.test_dir, class_dir, np.random.choice(os.listdir(os.path.join(test_dir, class_dir))))
# Preprocess the image
image = preprocess_image(image_file)
# Get the current joint angles and velocities
# Combine the image and joint states into an observation
observation = {"image": image}
return observation
def reset(self):
self.current_step = 0
# Generate a new observation as the initial state
observation = self._get_observation()
self.current_observation = observation
# Return the initial observation
return observation
def step(self, action):
# Take action in environment and return new observation, reward, and done flag
self.current_step += 1
new_state = self.current_observation + action
features = self.model.predict(new_state[None])[0]
reward = self._compute_reward(new_state, self.current_observation, action)
done = (self.current_step >= self.max_steps)
self.current_observation = new_state
return features.reshape(-1), reward, done, {}
def _compute_reward(self, new_state, previous_state, action):
# Compute the reward based on the new and previous states and the action taken
reward = np.sum(np.abs(new_state - previous_state))
# Return the reward
return reward
def render(self, mode='human'):
# Render environment
pass
# Define RL agent
class Agent:
def __init__(self, env):
self.env = env
self.observation_space = env.observation_space
self.action_space = env.action_space
self.policy_network = self._build_policy_network()
def _build_policy_network(self):
inputs = tf.keras.layers.Input(shape=self.observation_space.shape)
# x = tf.keras.applications.resnet_v2.preprocess_input(inputs)
resnet = model
for layer in resnet.layers[:-5]:
layer.trainable = False
x = resnet(inputs)
x = tf.keras.layers.Flatten()(x)
x = tf.keras.layers.Dense(64, activation='relu')(x)
outputs = tf.keras.layers.Dense(self.action_space.shape[0], activation='tanh')(x)
policy_network = tf.keras.models.Model(inputs=inputs, outputs=outputs)
return policy_network
def choose_action(self, observation):
# Use the policy network to generate an action
action = self.policy_network.predict([state])[0]
# Clip the action value to be within the valid range
action = np.clip(action, self.action_space.low, self.action_space.high)
# Return the action
return action
def train(self, iterations):
for i in range(iterations):
observation = self.env.reset()
done = False
while not done:
action = self.choose_action(observation)
new_observation, reward, done, _ = self.env.step(action)
# Update policy network using RL algorithm
pass
import gym
import numpy as np
# Create an instance of the CustomEnv class
env = CustomEnv()
# Create an instance of the Agent class
agent = Agent(env)
# Define the training loop
num_episodes = 1000
max_steps_per_episode = 1000
batch_size = 16
discount_factor = 0.99
optimizer = tf.keras.optimizers.Adam(learning_rate=0.001)
for episode in range(num_episodes):
# Reset the environment for the start of the episode
state = env.reset()
episode_reward = 0
episode_states = []
episode_actions = []
episode_rewards = []
for step in range(max_steps_per_episode):
# Choose an action based on the current state
action = agent.choose_action(state)
# Take a step in the environment based on the chosen action
new_state, reward, done, info = env.step(action)
# Compute the reward for the current step
previous_state = state if step == 0 else episode_states[-1]
step_reward = agent._compute_reward(new_state, previous_state, action)
# Add the state, action, and reward to the episode history
episode_states.append(state)
episode_actions.append(action)
episode_rewards.append(step_reward)
episode_reward += step_reward
# Update the current state
state = new_state
# If the episode is over, break out of the loop
if done:
break
# Compute the discounted rewards for each step in the episode
episode_discounted_rewards = np.zeros_like(episode_rewards)
cumulative_reward = 0
for i in reversed(range(len(episode_rewards))):
cumulative_reward = episode_rewards[i] + discount_factor * cumulative_reward
episode_discounted_rewards[i] = cumulative_reward
# Normalize the discounted rewards
episode_discounted_rewards -= np.mean(episode_discounted_rewards)
episode_discounted_rewards /= np.std(episode_discounted_rewards)
# Train the policy network using the episode history
num_batches = len(episode_states) // batch_size
for batch in range(num_batches):
batch_start = batch * batch_size
batch_end = (batch + 1) * batch_size
# Get the batch of states, actions, and discounted rewards
batch_states = np.array(episode_states[batch_start:batch_end])
batch_actions = np.array(episode_actions[batch_start:batch_end])
batch_discounted_rewards = np.array(episode_discounted_rewards[batch_start:batch_end])
# Compute the loss and gradients
with tf.GradientTape() as tape:
logits = agent.policy_network(batch_states)
loss = -tf.reduce_mean(batch_discounted_rewards * tf.math.log(logits + 1e-10) + (1 - batch_discounted_rewards) * tf.math.log(1 - logits + 1e-10))
gradients = tape.gradient(loss, agent.policy_network.trainable_variables)
# Apply the gradients to the optimizer
optimizer.apply_gradients(zip(gradients, agent.policy_network.trainable_variables))
# Print the episode reward every 10 episodes
if (episode + 1) % 10 == 0:
print(f"Episode {episode + 1}, Reward: {episode_reward}")
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) Cell In[110], line 27 23 episode_rewards = [] 25 for step in range(max_steps_per_episode): 26 # Choose an action based on the current state ---> 27 action = agent.choose_action(state) 29 # Take a step in the environment based on the chosen action 30 new_state, reward, done, info = env.step(action) Cell In[109], line 24, in Agent.choose_action(self, observation) 22 def choose_action(self, observation): 23 # Use the policy network to generate an action ---> 24 action = self.policy_network.predict([state])[0] 26 # Clip the action value to be within the valid range 27 action = np.clip(action, self.action_space.low, self.action_space.high) File ~/miniconda3/envs/hf/lib/python3.10/site-packages/keras/utils/traceback_utils.py:67, in filter_traceback.<locals>.error_handler(*args, **kwargs) 65 except Exception as e: # pylint: disable=broad-except 66 filtered_tb = _process_traceback_frames(e.__traceback__) ---> 67 raise e.with_traceback(filtered_tb) from None 68 finally: 69 del filtered_tb File /tmp/__autograph_generated_filegpzrq7x9.py:15, in outer_factory.<locals>.inner_factory.<locals>.tf__predict_function(iterator) 13 try: 14 do_return = True ---> 15 retval_ = ag__.converted_call(ag__.ld(step_function), (ag__.ld(self), ag__.ld(iterator)), None, fscope) 16 except: 17 do_return = False ValueError: in user code: File "/home/priteshk/miniconda3/envs/hf/lib/python3.10/site-packages/keras/engine/training.py", line 1845, in predict_function * return step_function(self, iterator) File "/home/priteshk/miniconda3/envs/hf/lib/python3.10/site-packages/keras/engine/training.py", line 1834, in step_function ** outputs = model.distribute_strategy.run(run_step, args=(data,)) File "/home/priteshk/miniconda3/envs/hf/lib/python3.10/site-packages/keras/engine/training.py", line 1823, in run_step ** outputs = model.predict_step(data) File "/home/priteshk/miniconda3/envs/hf/lib/python3.10/site-packages/keras/engine/training.py", line 1791, in predict_step return self(x, training=False) File "/home/priteshk/miniconda3/envs/hf/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler raise e.with_traceback(filtered_tb) from None File "/home/priteshk/miniconda3/envs/hf/lib/python3.10/site-packages/keras/engine/input_spec.py", line 264, in assert_input_compatibility raise ValueError(f'Input {input_index} of layer "{layer_name}" is ' ValueError: Input 0 of layer "model_57" is incompatible with the layer: expected shape=(None, 224, 224, 3), found shape=(32, 224, 3)
preds_probs = model.predict(test_data)
2023-03-28 16:24:20.320572: I tensorflow/stream_executor/cuda/cuda_dnn.cc:384] Loaded cuDNN version 8700
5/353 [..............................] - ETA: 13s
2023-03-28 16:24:24.101015: I tensorflow/stream_executor/cuda/cuda_blas.cc:1786] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
239/353 [===================>..........] - ETA: 4s
2023-03-28 16:24:33.165492: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
251/353 [====================>.........] - ETA: 3s
2023-03-28 16:24:33.585175: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile 2023-03-28 16:24:33.585680: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
318/353 [==========================>...] - ETA: 1s
2023-03-28 16:24:36.221128: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
353/353 [==============================] - 23s 41ms/step
preds_probs
array([[5.57500064e-01, 1.49156470e-02, 3.95028167e-07, ...,
4.03020772e-09, 1.92685297e-06, 7.63295027e-09],
[8.00372899e-01, 6.03676625e-02, 9.12816540e-05, ...,
4.56398884e-05, 1.27164649e-05, 1.05896497e-05],
[9.99792397e-01, 1.87173882e-05, 1.92202324e-10, ...,
1.31564887e-12, 1.08445420e-04, 1.29400293e-11],
...,
[3.80474596e-13, 2.90768583e-08, 4.39055185e-07, ...,
3.23493628e-11, 5.69656819e-08, 9.99963522e-01],
[1.64871852e-10, 4.19621711e-06, 4.79005530e-06, ...,
1.58595073e-03, 1.12237494e-07, 9.04017329e-01],
[3.26277561e-10, 1.61820171e-05, 5.30638536e-05, ...,
1.70158764e-05, 3.15923759e-10, 9.24969256e-01]], dtype=float32)
prediction = np.argmax(preds_probs, axis = 1)
prediction
array([ 0, 0, 0, ..., 24, 24, 24])
test_loss, test_accuracy = model.evaluate(test_data, verbose=0)
print("Loss on test set: {}".format(test_loss))
print("Accuracy on test set: {:.4f}%".format(test_accuracy*100))
2023-03-28 16:24:57.769452: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile 2023-03-28 16:24:58.232488: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile 2023-03-28 16:24:58.232669: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile 2023-03-28 16:25:01.064482: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
Loss on test set: 0.4836946427822113 Accuracy on test set: 86.9488%
from helper_functions import make_confusion_matrix
# To get out test labels we need to unravel out test_data BatchDataset
y_label = []
for images, labels in test_data.unbatch():
y_label.append(labels.numpy().argmax()) # currently test labels look like: [0, 0, 0, 0, 1 ... , 0, 0]
y_label[:10] # look at first 10
2023-03-28 16:25:07.967853: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile 2023-03-28 16:25:08.195947: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile 2023-03-28 16:25:08.203304: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile 2023-03-28 16:25:09.665524: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
len(y_label)
5647
make_confusion_matrix(y_true=y_label, y_pred=prediction, classes=test_data.class_names, figsize=(100, 100))
from sklearn.metrics import classification_report
print(classification_report(y_true=y_label, y_pred=prediction))
precision recall f1-score support
0 0.86 0.90 0.88 231
1 0.76 0.64 0.70 232
2 0.96 0.98 0.97 229
3 0.94 0.92 0.93 231
4 0.88 0.94 0.91 231
5 0.73 0.80 0.76 229
6 0.90 0.75 0.82 229
7 0.89 0.69 0.78 194
8 0.81 0.90 0.85 231
9 0.82 0.88 0.85 231
10 0.96 0.92 0.94 231
11 0.88 0.88 0.88 231
12 0.76 0.85 0.80 230
13 0.93 0.92 0.92 229
14 0.86 0.87 0.87 157
15 0.95 0.82 0.88 231
16 0.73 0.89 0.80 232
17 0.90 0.89 0.89 232
18 0.89 0.88 0.88 232
19 0.95 0.90 0.92 231
20 0.75 0.89 0.81 215
21 0.96 0.91 0.94 232
22 0.94 0.90 0.92 232
23 0.93 0.91 0.92 232
24 0.92 0.91 0.91 232
accuracy 0.87 5647
macro avg 0.87 0.87 0.87 5647
weighted avg 0.87 0.87 0.87 5647
# Get a dictionary of classification report
classification_report_dict = classification_report(y_true=y_label, y_pred=prediction, output_dict=True)
class_names = test_data.class_names
# Create empty dictionary
class_f1_scores = {}
accuracy = None
# Loop through classification report
for k, v in classification_report_dict.items():
if k == "accuracy": # Stop once we get to accuracy key
accuracy = v
break
else:
# Add class names and F1 scores o new dictionaty
class_f1_scores[class_names[int(k)]] = v["f1-score"]
class_f1_scores
{'Asian Green Bee-Eater': 0.8752642706131079,
'Brown-Headed Barbet': 0.6962616822429907,
'Cattle Egret': 0.9696969696969697,
'Common Kingfisher': 0.9301310043668122,
'Common Myna': 0.9056603773584906,
'Common Rosefinch': 0.760914760914761,
'Common Tailorbird': 0.8162291169451074,
'Coppersmith Barbet': 0.7790697674418604,
'Forest Wagtail': 0.8536082474226804,
'Gray Wagtail': 0.8458333333333333,
'Hoopoe': 0.9380530973451326,
'House Crow': 0.8806941431670282,
'Indian Grey Hornbill': 0.8049281314168377,
'Indian Peacock': 0.9210526315789475,
'Indian Pitta': 0.8670886075949368,
'Indian Roller': 0.8796296296296297,
'Jungle Babbler': 0.8015564202334631,
'Northern Lapwing': 0.8917748917748919,
'Red-Wattled Lapwing': 0.8845315904139432,
'Ruddy Shelduck': 0.9247787610619468,
'Rufous Treepie': 0.8110403397027601,
'Sarus Crane': 0.9380530973451328,
'White Wagtail': 0.9183222958057395,
'White-Breasted Kingfisher': 0.9150326797385621,
'White-Breasted Waterhen': 0.9110629067245121}
# turn f1 scores into dataframe for visualization
import pandas as pd
f1_scores = pd.DataFrame({"class_names": list(class_f1_scores.keys()),
"f1-score": list(class_f1_scores.values())}).sort_values("f1-score", ascending=False)
We compare F1 scores (precision & recall accuracies) of all classes which conclude that Cattle Egret has the highest F1 score at 0.97 and conversely Brown-Headed Barbet has the lowest F1 score at 0.70
import matplotlib.pyplot as plt
fig, ax = plt.subplots(figsize=(25, 8))
scores = ax.bar(range(len(f1_scores)), f1_scores["f1-score"].values)
ax.set_xticks(range(len(f1_scores)))
ax.set_xticklabels(f1_scores["class_names"], rotation=90, ha='right')
ax.set_ylabel("F1-score")
plt.axhline(y=accuracy, linewidth=2, color='r')
ax.set_title("F1-score for 25 Bird classes with Accuracy Threshold line")
ax.bar_label(scores, fmt="%.2f");
Steps:
# Create a function to load and prepare images
def load_and_prep_image(filename, img_size=224, scale=True):
"""
reads in an image form filename, turns it into a tensor and reshapes into specified shape
Args:
filename (str): path to target image
img_shape (int): height/width dimentions of target image size
scale (bool): scale pixel values from 0-255 to 0-1 or not
Returns:
Image tensor of shape (img_shape, img_shape, 3)
"""
# Read in the image
img = tf.io.read_file(filename=filename)
# Decode image into tensor
img = tf.io.decode_image(img, channels=3)
# Resize the image
img = tf.image.resize(img, [img_size, img_size])
# Scaling
if scale:
# Rescale the image
return img/255.
else:
return img
# visualize test images
# Make preds on series of random images
import os
import random
plt.figure(figsize=(17, 10))
for i in range(3):
# Choose a random image from random class
class_name = random.choice(class_names)
filename = random.choice(os.listdir(test_dir + "/" + class_name))
filepath = test_dir + "/" + class_name + "/" + filename
# Load the images and make predictions
img = load_and_prep_image(filepath, scale=False)
img_expanded = tf.expand_dims(img, axis=0)
# pred_prob = model.predict(img_expanded) # get perdiction probability array
pred_prob = model.predict(img_expanded) # get perdiction probability array
pred_class = class_names[pred_prob.argmax()] # get highest prediction probability index
# Plot the images
plt.subplot(1, 3, i+1)
plt.imshow(img/255)
if class_name == pred_class: # if predicted class matches, truth class, make text green
title_color = "g"
else:
title_color = "r"
plt.title(f"Actual class: {class_name}, pred: {pred_class}, prob: {pred_prob.max():.2f}", c=title_color)
plt.axis(False);
1/1 [==============================] - 0s 106ms/step 1/1 [==============================] - 0s 26ms/step 1/1 [==============================] - 0s 30ms/step
We try to identify the images where model wrongly predicted a bird with high probability and we conclude the reason being low contrast, camouflage of birds with the background, shadows, etc.
# get all of the files from test_data
filepaths = []
for filepath in test_data.list_files("testing_set/*/*.jpg", shuffle=False):
filepaths.append(filepath.numpy())
filepaths[:10]
[b'testing_set/Asian Green Bee-Eater/ML100845971.jpg', b'testing_set/Asian Green Bee-Eater/ML102065651.jpg', b'testing_set/Asian Green Bee-Eater/ML108389751.jpg', b'testing_set/Asian Green Bee-Eater/ML108490471.jpg', b'testing_set/Asian Green Bee-Eater/ML109116691.jpg', b'testing_set/Asian Green Bee-Eater/ML112055301.jpg', b'testing_set/Asian Green Bee-Eater/ML118156651.jpg', b'testing_set/Asian Green Bee-Eater/ML118645851.jpg', b'testing_set/Asian Green Bee-Eater/ML119513941.jpg', b'testing_set/Asian Green Bee-Eater/ML120026161.jpg']
# Create a dataframe of different parameters
pred_df = pd.DataFrame({"img_path": filepaths,
"y_true": y_label,
"y_pred": prediction,
"pred_conf": preds_probs.max(axis=1), # Get the maximum prediction probability value
"y_true_classname": [class_names[i] for i in y_label],
"y_pred_classname": [class_names[i] for i in prediction]})
pred_df
| img_path | y_true | y_pred | pred_conf | y_true_classname | y_pred_classname | |
|---|---|---|---|---|---|---|
| 0 | b'testing_set/Asian Green Bee-Eater/ML10084597... | 0 | 0 | 0.966927 | Asian Green Bee-Eater | Asian Green Bee-Eater |
| 1 | b'testing_set/Asian Green Bee-Eater/ML10206565... | 0 | 0 | 0.999225 | Asian Green Bee-Eater | Asian Green Bee-Eater |
| 2 | b'testing_set/Asian Green Bee-Eater/ML10838975... | 0 | 12 | 0.334575 | Asian Green Bee-Eater | Indian Grey Hornbill |
| 3 | b'testing_set/Asian Green Bee-Eater/ML10849047... | 0 | 0 | 0.999982 | Asian Green Bee-Eater | Asian Green Bee-Eater |
| 4 | b'testing_set/Asian Green Bee-Eater/ML10911669... | 0 | 0 | 0.958633 | Asian Green Bee-Eater | Asian Green Bee-Eater |
| ... | ... | ... | ... | ... | ... | ... |
| 4509 | b'testing_set/White-Breasted Waterhen/ML862254... | 24 | 24 | 0.725891 | White-Breasted Waterhen | White-Breasted Waterhen |
| 4510 | b'testing_set/White-Breasted Waterhen/ML870705... | 24 | 24 | 0.999892 | White-Breasted Waterhen | White-Breasted Waterhen |
| 4511 | b'testing_set/White-Breasted Waterhen/ML872301... | 24 | 24 | 0.999891 | White-Breasted Waterhen | White-Breasted Waterhen |
| 4512 | b'testing_set/White-Breasted Waterhen/ML912482... | 24 | 24 | 0.970743 | White-Breasted Waterhen | White-Breasted Waterhen |
| 4513 | b'testing_set/White-Breasted Waterhen/ML923868... | 24 | 24 | 0.999579 | White-Breasted Waterhen | White-Breasted Waterhen |
4514 rows × 6 columns
# Find which predictions are wrong
pred_df["pred_correct"] = pred_df["y_true"] == pred_df["y_pred"]
pred_df.head()
| img_path | y_true | y_pred | pred_conf | y_true_classname | y_pred_classname | pred_correct | |
|---|---|---|---|---|---|---|---|
| 0 | b'testing_set/Asian Green Bee-Eater/ML10084597... | 0 | 0 | 0.966927 | Asian Green Bee-Eater | Asian Green Bee-Eater | True |
| 1 | b'testing_set/Asian Green Bee-Eater/ML10206565... | 0 | 0 | 0.999225 | Asian Green Bee-Eater | Asian Green Bee-Eater | True |
| 2 | b'testing_set/Asian Green Bee-Eater/ML10838975... | 0 | 12 | 0.334575 | Asian Green Bee-Eater | Indian Grey Hornbill | False |
| 3 | b'testing_set/Asian Green Bee-Eater/ML10849047... | 0 | 0 | 0.999982 | Asian Green Bee-Eater | Asian Green Bee-Eater | True |
| 4 | b'testing_set/Asian Green Bee-Eater/ML10911669... | 0 | 0 | 0.958633 | Asian Green Bee-Eater | Asian Green Bee-Eater | True |
# Sort dataframe to have most wrong predictions at top
top_100_wrong = pred_df[pred_df["pred_correct"] == False].sort_values("pred_conf", ascending=False)[:100]
top_100_wrong.head(20)
| img_path | y_true | y_pred | pred_conf | y_true_classname | y_pred_classname | pred_correct | |
|---|---|---|---|---|---|---|---|
| 191 | b'testing_set/Brown-Headed Barbet/ML119679471.... | 1 | 7 | 0.999370 | Brown-Headed Barbet | Coppersmith Barbet | False |
| 2099 | b'testing_set/House Crow/ML256308541.jpg' | 11 | 4 | 0.999160 | House Crow | Common Myna | False |
| 1135 | b'testing_set/Common Tailorbird/ML149453221.jpg' | 6 | 9 | 0.998646 | Common Tailorbird | Gray Wagtail | False |
| 2331 | b'testing_set/Indian Grey Hornbill/ML63915191.... | 12 | 8 | 0.998476 | Indian Grey Hornbill | Forest Wagtail | False |
| 565 | b'testing_set/Common Kingfisher/ML152225741.jpg' | 3 | 23 | 0.998271 | Common Kingfisher | White-Breasted Kingfisher | False |
| 231 | b'testing_set/Brown-Headed Barbet/ML196420071.... | 1 | 16 | 0.996154 | Brown-Headed Barbet | Jungle Babbler | False |
| 2917 | b'testing_set/Jungle Babbler/ML282513391.jpg' | 16 | 6 | 0.995713 | Jungle Babbler | Common Tailorbird | False |
| 4169 | b'testing_set/White-Breasted Kingfisher/ML1646... | 23 | 18 | 0.995000 | White-Breasted Kingfisher | Red-Wattled Lapwing | False |
| 806 | b'testing_set/Common Myna/ML221310241.jpg' | 4 | 24 | 0.994229 | Common Myna | White-Breasted Waterhen | False |
| 1338 | b'testing_set/Coppersmith Barbet/ML176468941.jpg' | 7 | 11 | 0.993855 | Coppersmith Barbet | House Crow | False |
| 1306 | b'testing_set/Coppersmith Barbet/ML131700641.jpg' | 7 | 14 | 0.993511 | Coppersmith Barbet | Indian Pitta | False |
| 3887 | b'testing_set/Sarus Crane/ML350856521.jpg' | 21 | 2 | 0.990375 | Sarus Crane | Cattle Egret | False |
| 4109 | b'testing_set/White Wagtail/ML498687651.jpg' | 22 | 8 | 0.990031 | White Wagtail | Forest Wagtail | False |
| 361 | b'testing_set/Brown-Headed Barbet/ML91802381.jpg' | 1 | 5 | 0.988294 | Brown-Headed Barbet | Common Rosefinch | False |
| 1279 | b'testing_set/Common Tailorbird/ML81698581.jpg' | 6 | 16 | 0.984620 | Common Tailorbird | Jungle Babbler | False |
| 1492 | b'testing_set/Forest Wagtail/ML212905501.jpg' | 8 | 9 | 0.983442 | Forest Wagtail | Gray Wagtail | False |
| 2683 | b'testing_set/Indian Roller/ML120308811.jpg' | 15 | 5 | 0.980947 | Indian Roller | Common Rosefinch | False |
| 4469 | b'testing_set/White-Breasted Waterhen/ML351459... | 24 | 20 | 0.979722 | White-Breasted Waterhen | Rufous Treepie | False |
| 1331 | b'testing_set/Coppersmith Barbet/ML155867261.jpg' | 7 | 20 | 0.979433 | Coppersmith Barbet | Rufous Treepie | False |
| 1190 | b'testing_set/Common Tailorbird/ML205140771.jpg' | 6 | 9 | 0.975412 | Common Tailorbird | Gray Wagtail | False |
# Visualize wrong predictions
images_to_view = 9
start_index = 10
plt.figure(figsize=(30, 20))
for i, row in enumerate(top_100_wrong[start_index:start_index + images_to_view].itertuples()):
plt.subplot(3, 3, i+1)
img = load_and_prep_image(row[1], scale=False)
_, _, _, _, pred_prob, y_true_classname, y_pred_classname, _ = row # skipping few values
plt.imshow(img/255)
plt.title(f"actual: {y_true_classname}, pred: {y_pred_classname}, prob: {pred_prob}")
plt.axis(False);
custom_bird_images = ["custom_images/" + img_path for img_path in os.listdir("custom_images")]
custom_bird_images
['custom_images/hoopoe.jpg', 'custom_images/Indian Pitta.jpg', 'custom_images/crow.jpg', 'custom_images/Common tailorbird Orthotomus sutorius.jpg', 'custom_images/Common myna.jpg']
# make predictions and plot custom food images
for img in custom_bird_images:
img = load_and_prep_image(img, scale=False)
pred_prob = model.predict(tf.expand_dims(img, axis=0))
pred_class = class_names[pred_prob.argmax()]
# Plot the information
plt.figure()
plt.imshow(img/255.)
plt.title(f"pred: {pred_class}, prob: {pred_prob.max():.2f}")
plt.axis(False);
1/1 [==============================] - 0s 108ms/step 1/1 [==============================] - 0s 29ms/step 1/1 [==============================] - 0s 65ms/step 1/1 [==============================] - 0s 24ms/step 1/1 [==============================] - 0s 26ms/step
We use tensorboard to compare the epoch losses & epoch accuracies of base model and fine-tuned model
%load_ext tensorboard
!tensorboard dev auth revoke
2023-03-28 16:08:21.014981: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2023-03-28 16:08:22.259226: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node Your kernel may have been built without NUMA support. 2023-03-28 16:08:22.312230: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node Your kernel may have been built without NUMA support. 2023-03-28 16:08:22.312308: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node Your kernel may have been built without NUMA support. Logged out of uploader.
!tensorboard dev upload --one_shot --logdir transfer_learning/
2023-03-28 16:08:26.376638: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2023-03-28 16:08:27.540871: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node Your kernel may have been built without NUMA support. 2023-03-28 16:08:27.589219: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node Your kernel may have been built without NUMA support. 2023-03-28 16:08:27.589297: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node Your kernel may have been built without NUMA support. ***** TensorBoard Uploader ***** This will upload your TensorBoard logs to https://tensorboard.dev/ from the following directory: transfer_learning/ This TensorBoard will be visible to everyone. Do not upload sensitive data. Your use of this service is subject to Google's Terms of Service <https://policies.google.com/terms> and Privacy Policy <https://policies.google.com/privacy>, and TensorBoard.dev's Terms of Service <https://tensorboard.dev/policy/terms/>. This notice will not be shown again while you are logged into the uploader. To log out, run `tensorboard dev auth revoke`. Continue? (yes/NO)
After fine-tuning, we are seeing somewhat improved accuracy than base model. However, as per trend, we can see that with much longer training, it will be able to improve accuracy to much higher level.
Currently, we got 85.05% accuracy with our base model and after fine-tuning bottom 5 layers and 5 epochs, we were able to achieve accuracy of 86.96% which can be improved significantly if we unfreeze and fine-tune more layers and train with more epochs
Next step would be to implement siamese network to add more classes to our classification model(as a part of our Project-2).
Idea is to implement siamese network using this dataset: https://www.kaggle.com/datasets/gpiosenka/100-bird-species
This will add 510 classes to the model